COV Pipeline context and parts that have already been run through
Note: The COV data is stored on microcenter at /media/gqc/T7/cctv/COV/Data/Video_Lists.
| Notebook Order | Notebook Name | Python File Name | Run for COV |
|---|---|---|---|
| 1 | cctv-video_list-creator.ipynb | video_list_creator.py | Was not run through video_list-creator since we did not have actual COV videos on hand. The videos table was populated using the video_lists csv which are present in COV/Data/Video_Lists |
| 2 | metadata_DB_extract_inspection_condition_data.ipynb | extract_inspection_condition_data.py | Was not run through metadata_DB_extract_inspection_condition_data since I(Dhananjay) did not have |
| 3 | cctv-import_supplementary_data.ipynb | import_suppl_data.py | This module import distance regions, video-types, Condition_code_standard, and all_conditions CSV files into Video_DB as separate tables. It also updates videos table's video-type column. |
| 4 | cctv-frame-extraction.ipynb | frame_extractor.py | This module extracts frames from a list of videos and saves them under a zip file. |
| 5 | cctv-full-frame-stitch-and-Azure-OCR.ipynb | stitch_and_ocr.py | This will do the following:Run Azure OCR on 6 stacked full frames,The OCR results will be saved to OCR_JSON column in the frames table which can be used for blurring in the next stage. |
| 6 | cctv-save-blurred-frames.ipynb | blur_frames.py | This module takes in the extracted frames and the CSV files generated by cctv-frame-extraction.ipynb utility and recognize the text present in the frames. |
| 7 | cctv-extract-distance-values.ipynb | extract_distance_values.py | This module takes in the OCR_JSON column in the frames table containing OCR results and extracts the distance values |
| 8 | cctv-distance-value-post-processing.ipynb | distance_postprocessor.py | This module processes the distance values detected by OCR to remove outliers and fill in the frames without distances by linear interpolation and extrapolation. |
| 9 | cctv-extract-condition_codes.ipynb | extract_condition_codes.py | This module takes in the OCR_JSON column in the frames table containing OCR results and extracts condition codes. |
| 10 | cctv_SD1_label_images.ipynb | SD1_label_images.py | This notebook is used to assign condition codes to each frame. |
| 11 | cctv-prepare-cctv-training-csv.ipynb | training_data_splitter.py | This module creates the correct data split between training, validation and test and writes the training, validation, and test into the sqlite3 database. |